The present invention relates to the field of three-dimensional image capture and in particular to techniques for modulating the reflected light in order to extract phase images and for capturing a color texture image in conjunction with the phase image.
Distance (or depth) information from a camera to objects in a scene can be obtained by using a scannerless range imaging system having a modulated illumination source and a modulated image receiver. In a method and apparatus described in U.S. Pat. No. 4,935,616 (and further described in the Sandia Lab News, vol. 46, No. 19, Sep. 16, 1994), a scannerless range imaging system uses either an amplitude-modulated high-power laser diode or an array of amplitude-modulated light emitting diodes (LEDs) to simultaneously illuminate a target area. Conventional optics confine the target beam and image the target onto a receiver, which includes an integrating detector array sensor having hundreds of elements in each dimension. The range to a target is determined by measuring the phase shift of the reflected light from the target relative to the amplitude-modulated carrier phase of the transmitted light. To make this measurement, the gain of an image intensifier (in particular, a micro-channel plate) within the receiver is modulated at the same frequency as the transmitter, so the amount of light reaching the sensor (a charge-coupled device) is a function of the range-dependent phase difference. A second image is then taken without receiver or transmitter modulation and is used to eliminate non-range-carrying intensity information. Both captured images are registered spatially, and a digital processor is used to extract range data from these two frames. Consequently, the range associated with each pixel is essentially measured simultaneously across the whole scene.
The scannerless range imaging system described above utilizes an image intensifier (specifically, a micro-channel plate) of the type produced by Litton Industries. The primary purpose of the intensifier is to provide a reference frequency to operate upon the modulated light signal from the illuminator that is reflected from the target. By modulating the gain of the image intensifier the reflected, modulated light signal is multiplied by the intensifier gain and constructive and destructive interference is established. A primary application of the scannerless range imaging system is to enable a method of creating a virtual three-dimensional environment from photographs. While range data is an important part of this application, a so-called texture image is also needed. The texture image should ideally be captured with identical optical properties as the range data to assure proper registration between range and texture values. Furthermore, having a color texture image is highly desirable for many practical and commercial applications.
A drawback of methods using an image intensifier is that color information is lost. Unfortunately for color applications, an image intensifier operates by converting photonic energy into a stream of electrons, amplifying the energy of the electrons and then converting the electrons back into photonic energy via a phosphor plate. One consequence of this process is that color information is lost. Since color is a useful property of images for many applications, a means of acquiring the color information that is registered along with the range information is extremely desirable.
One approach to acquiring color is to place a dichromatic mirror in the optical path before the micro-channel-plate. Following the mirror a separate image capture plane (i.e., a separate image sensor) is provided for the range portion of the camera and another image capture plane (another sensor) is provided for the color texture capture portion of the camera. This is the approach taken by 3DV Technology with their Z-Cam product. Besides the added expense of two image capture devices, there are additional drawbacks in the need to register the two image planes precisely, together with alignment of the optical paths. Another difficulty is collating image pairs gathered by different sources.
Recognizing that the system described in the ""616 patent may be implemented in relation to a normal camera system, and, in particular, that a standard camera system may be converted into a range capture system by modifying its optical system, another approach is to employ interchangeable optical assemblies: one optical assembly for the phase image portion and a separate optical element for the color texture image portion. This approach is described in detail in commonly assigned copending application Ser. No. 09/451,823, entitled xe2x80x9cMethod and Apparatus for a Color Scannerless Range Image Systemxe2x80x9d and filed Nov. 30, 1999 in the names of Lawrence A. Ray, Louis R. Gabello and Kenneth J. Repich. The drawback of this approach is the need to switch lenses and the possible misregistration that might occur due to the physical exchange of lens elements. There is an additional drawback in the time required to swap the two optical assemblies, and the effect that may have on the spatial coincidence of the images.
In commonly-assigned, copending U.S. patent application Ser. No. 09/572,522, now U.S. Pat. No. 6,349,174 B1 entitled xe2x80x9cMethod and Apparatus for a Color Scannerless Range Imaging Systemxe2x80x9d and filed May 17, 2000 in the names of Lawrence A. Ray and Louis R. Gabello, a beamsplitter located in the primary optical path separates the reflected image light into two channels, a first channel including an infrared component and a second channel including a color texture component, whereby one of the channels traverses a secondary optical path distinct from the primary path. A modulating element, i.e., an intensifier, is operative in the first channel to receive the infrared component and a modulating signal, and to generate a processed infrared component with phase data indicative of range information. An optical network is provided in the secondary optical path for recombining the secondary optical path into the primary optical path such that the processed infrared component and the color texture component are directed to the image responsive element. This technique eliminates the requirement for two image capture planes, as well as for interchangeable optical assemblies, and allows the operator to collect a full range map with texture with a single exposure activation.
In addition to the loss of color information, and the consequent necessity to devise techniques as described above to overcome this drawback, the image intensifier is a costly part and, in addition, can be fragile. In order to reduce the cost of the scannerless range imaging system, a less expensive alternative technology would be attractive. Since a primary purpose of the image intensifier is to act as a modulating shutter, an alternative technology will have to perform this task. What is needed is an alternative technology that would avoid the aforementioned limitations; in addition, it would be desirable to capture ranging information without sacrificing color information that would otherwise be available for capture.
It is an object of the invention to provide an alternative technology to an image intensifier for the receiver modulation function in a scannerless range imaging system.
It is a further object of the invention to capture a color texture image as well as one or more phase images on the same image plane for each point on the image.
The present invention is directed to achieving these objectives while overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, a scannerless range imaging system for capturing range information of a scene includes an illumination system and an electromechanical light modulator. The illumination system illuminates objects in the scene with modulated illumination of a redetermined modulation frequency, whereby the modulated illumination reflected from objects in the scene incorporates a phase delay corresponding to the distance of the objects from the range imaging system. The electromechanical light modulator, which is positioned in an optical path of the modulated illumination reflected from the object, operates at a reference frequency that corresponds to the predetermined modulation frequency and accordingly modulates the modulated illumination reflected from the object, thereby generating an image from the interference between the reference frequency and the reflected modulated illumination. This captured image, which is thereafter referred to as a phase image, is used to derive range data. An image capture section, also positioned in the optical path of the modulated illumination reflected from the object, captures the phase image.
Since the electromechanical light modulator operates via modulating elements having reflective surfaces, the system further includes an optical system having a mirror element that deflects the reflected modulated illumination upon the reflective surfaces of the electromechanical light modulator and redirects the phase image reflected from the reflective surfaces of the electromechanical light modulator to the image capture section. A color image may be captured by moving the optical system out of the optical path such that reflected illumination from the object will pass directly to the image capture section without contacting the electromechanical light modulator. A preferred electromechanical light modulator is an electromechanical grating.
Consequently, an advantage of the invention is that it provides both an alternative to the intensifier and a simplified technique for capturing a color texture image as well as one or more phase images. A further advantage of the invention is that it eliminates the need for an expensive component, i.e., the intensifier, with a device that is significantly less expensive and less fragile. The electromechanical grating is also lighter in weight and more compact than a micro-channel plate, and uses a lower operating voltage. Moreover, the system is able to capture a color image in addition to the phase images without the sort of clever work-arounds shown in the prior art. The system is also able to operate in a continuous modulation mode or in a pulse mode.
These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiments and appended claims, and by reference to the accompanying drawings.